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Community College Pathways' Program Success 


ABSTRACT 

Statway® is one of the Community College Pathways initiatives developed by the Carnegie Foundation 
for the Advancement of Teaching to promote students' progress beginning in developmental math to 
and through college math credit. Early descriptive results suggested that the Statway program has 
tripled the success rate for students in half the time to achieve college math credit. This study presents a 
more rigorous causal analysis of hypothesized program effects. We used a multilevel approach with 
propensity score matching to evaluate the effectiveness of Statway. Propensity score matching results 
are consistent with earlier descriptive findings. They replicate across two different cohorts and hold up 
for varying genders, race/ethnicity groups, and math placement levels. Directions for future work are 
also suggested. 
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Community College Pathways' Program Success: 

Assessing the First Two Years' Effectiveness of Statway° 

Community colleges are intended to provide educational opportunities that prepare citizens to lead 
productive economic lives and contribute as members of society. Every year, however, the hopes and 
dreams of hundreds of thousands of students are dashed, and their access to economically successful, 
self-sustaining lives is denied. Approximately 60 percent of incoming students are referred to at least 
one developmental math course, of which 80 percent will not earn college-level math credit even after 
three years (Bailey, Jeong, & Cho, 2010). These students lack command of the math that matters for 
living in an increasingly quantitative age and becoming critically engaged citizens. Without achieving 
college math credit, they cannot transfer into four-year degree programs or qualify for entry into 
preparation programs in a wide range of occupational-technical specialties. Without a doubt, this is one 
of the greatest social equity problems of our time (Cullinane & Treisman, 2010). 

To tackle this problem, the Community College Pathways (CCP) was developed and implemented 
through a Networked Improvement Communities (NICs) structure involving faculty members, 
researchers, designers, and content experts (Bryk, Gomez, & Grunow, 2011; Dolle, Gomez, Russell, & 

Bryk, 2013). NICs are scientific learning communities distinguished by four essential characteristics, 
which are (a) focused on a well specified common aim, (b) guided by a deep understanding of the 
problem and the system that produces it, (c) disciplined by the rigor of improvement science, and (d) 
networked to accelerate the development, testing, and refinement of interventions and their effective 
integration into varied educational contexts. The CCP consists of two different mathematics pathways, 
Statway° and Quantway . Statway is a one-year pathway focused on statistics, data analysis, and causal 
reasoning that combines college-level statistics with developmental math; Quantway focuses on 
quantitative reasoning -a general education mathematics requirement in many colleges. Both pathways 
seek to accelerate students' progress through developmental math by allowing them to earn college 
math credit within a single academic year. (For more details about the Pathways, see Strother, Van 
Campen, & Grunow, 2013; Van Campen, Sowers, & Strother, 2013). The current report focuses on 
Statway results from its first two years of implementation. 1 

Statway is designed to target students who are placed two or more levels below college-level math and serve 
students planning to transfer and continue further studies in humanities or social sciences. To promote their 
ambitious learning goals, the Statway instructional system employs three research-based principles: 

1. Productive struggle: students are more likely to retain what they learn when they expend effort 
solving problems that are within reach and grappling with key mathematical ideas that are 
comprehensible but not yet well-formed. Thus, each new concept is introduced with a rich 
problem that engages students' thinking and encourages this struggle to understand (Hiebert & 
Grouws, 2007; Schmidt & Bjork, 1992). 


1 The analyses of Quantway success are underway. 
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2. Explicit connections to concepts: mathematics instruction sometimes focuses on procedural 
competence at the cost of advancing real conceptual understanding. However, research 
suggests making explicit connections between mathematical or statistical facts, ideas, and 
procedures can improve both conceptual and procedural understanding (Boaler, 1998; Hiebert 
& Grouws, 2007). 

3. Deliberate practice: classroom and homework tasks are designed to overcome gaps in 
understanding, apply what is learned, and deepen facility with key concepts (Ericsson, 2008; 
Ericsson, Krampe, & Tescher-Romer, 1993). Deliberate practice eschews rote repetition for 
carefully sequenced problems developed to guide students toward deeper understanding of 
core concepts (Pashler, Rohrer, Cepeda, & Carpenter, 2007). 

Early findings from Year 1 (2011-2012) indicated Statway's success (Strother et al., 2013): Fifty-one 
percent of students successfully completed the full-year pathway, thereby earning college math credit 
(with a grade of C or higher). In previous years, only 15 percent of developmental math students 
achieved a similar level of success within two years of college enrollment. Data from Year 2 (2012-2013) 
established success rates comparable to those of Year 1 (Van Campen et al., 2013). These findings 
suggest that Statway tripled the success in half the time. 

Statway appears to have achieved extraordinary results. However, some may argue that this is merely 
the result of selection bias, where certain kinds of students may have been more likely to enroll in 
Statway, leading to more positive outcomes than there otherwise would have been. To address this 
issue, the main objective of the current study was to formulate a more rigorous causal test to evaluate 
the Statway's effectiveness. We used a propensity score matching technique to statistically reduce 
selection bias and accordingly increase the validity of causal inference (Rosenbaum & Rubin, 1983). 
Given the nature of our data (i.e., students nested within colleges), we employed a hierarchical linear 
modeling (HLM) approach (Raudenbush & Bryk, 2002) to obtain propensity scores (Hong & Raudenbush, 
2005, 2006). We then separately examined data from Year 1 and Year 2. 

A second objective was to track the academic outcomes of students one year after their enrollment in 
Statway. For this purpose, we compared college-level course credit accumulation between Statway and 
non-Statway matched groups in the subsequent year. 2 This comparative analysis was intended to 
determine whether Statway students continue to demonstrate success even after their Statway 
experience. 


2 This follow-up performance was examined only for Year 1; data from Year 2 were not available in this study. 
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METHOD 


Participants 

The CCP first launched during the 2011-2012 academic year. The first cohort of students began Statway 
in the fall of 2011. This initial cohort of students spanned 19 community colleges across five states. In 
total, 50 faculty members taught 55 sections of Statway with 1133 students enrolled (Strother et al., 
2013). The second cohort included a total of 1553 students enrolled in 77 sections of Statway taught by 
67 faculty members. Of the 19 community colleges that participated in Statway, all but one offered 
Statway in both Year 1 and Year 2 (Van Campen et al., 2013). 

Table 1 summarizes the demographic characteristics of Statway students. The vast majority of students 
placed at least two levels below a college-level math course, and almost half were also required to take 
at least one developmental reading course. Approximately 60 percent of the students were female, and 
less than one-third were raised in families where the mother held a two- or four-year college degree. 
Well over half of the students were minorities. 

Data Collection 

Institutional researchers from participating colleges provided background data on student 
characteristics, course enrollment and performance. For Year 1, we excluded two community colleges 
from the analyses. One college discontinued the program partway through the year because its district 
mandated an alternative developmental math program. The second college implemented Statway in a 
non-standard way. Consequently, the Year 1 analytic sample consisted of 928 Statway students from 17 
community colleges. For Year 2, we excluded four community colleges: two that implemented Statway 
in a non-standard way and two that failed to provide the institutional data necessary for propensity 
score matching. In sum, institutional data were available for 771 Statway students from 15 community 
colleges. All 15 of these colleges also offered Statway in Year 1. 

Study Design 

Figure 1 delineates the study design using Year 1 as an exemplar. The first objective in this study was to 
identify a group of students most comparable with Statway students. Defining an appropriate 
comparison group in this instance was a little more complex than might normally be the case. Statway is 
designed as an intensive course intended to fulfill requirements for both developmental math and 
college-level math within one year of continuous enrollment. Students starting two or more levels 
behind college-level math typically cannot accomplish this in one year if they follow a traditional 
program of study. They would need to be enrolled for one and a half to two years to meet the same 
benchmark. This led us to draw a comparison group consisting of students who began taking their 
developmental math course one year before their Statway counterparts and then compare both groups' 
outcomes at the end of the Statway year. Thus, comparison students had two years to achieve the same 
outcomes that Statway students accomplished in one year. For instance, the comparison group for 
students who began Statway in fall 2011 consisted of students who began developmental math in fall 
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2010. These two groups were then compared at the end of the spring 2012. Both groups were also 
followed for an additional year to examine subsequent course taking and credit accumulation. The same 
strategy was used to form a comparison group for the second cohort of students beginning Statway in 
fall 2012. 

To obtain propensity scores, we formulated a two-level model with a total of 44 student-level covariates 
including student background characteristics, course enrollment, and performance during the two years 
prior to fall 2010/2011 for the Year 1 cohort and during the two years prior to fall 2011/2012 for the 
Year 2 cohort. Tables 2 and 3 present all of the covariates used and their descriptive statistics for Year 1 
and Year 2, respectively. For some variables, students' information is recorded as "unknown," and not 
all colleges kept data on all variables used in the propensity score matching. For instance, there is a 
substantial number of unknown records for student placement levels, indicating that data are simply 
missing or students did not take a placement test. Missing GPAs correspond to students who did not 
take college-level courses or received grades that do not have an effect on their GPAs (i.e., W 
[Withdrawals] and I [Incompletes]). Accordingly, we assigned a code of -1 to students without GPA in 
each GPA variable. To factor these "missing" records in the propensity model, we formulated a dummy 
variable for each GPA and coded missing GPAs as 1, otherwise 0. 

In general, Statway students in Year 1 were more likely to be drawn from a second year or older cohort 
(i.e., not a first-year student cohort), a somewhat different race/ethnicity composition (notably less 
Hispanic students), and those placed two levels below college math. They were also much older and 
took more courses overall in the last two years prior to the Statway start term (except for college math 
courses). Year 2 included more full-time students and those who took less developmental non-math 
courses. 

We conducted propensity score matching separately for each college by applying a nearest neighbor 
matching algorithm (Rosenbaum & Rubin, 1985). This algorithm was appropriate for our study because 
we wanted to retain all Statway students and had a large pool of non-Statway students available for 
creating matches. We attempted to find up to five matches per Statway student (5:1 ratio matching) to 
maximize the best matches from the non-Statway student group while still maintaining precision (Ming 
& Rosenbaum, 2000). We also specified a caliper distance of up to 0.2 to reduce the risk of bad nearest 
neighbor matches based on recommendations in the literature (Austin, 2011; Rosenbaum & Rubin, 
1985). 

Next, as portrayed in the middle panel of Figure 1, we estimated the effectiveness of Statway (i.e., 
success rate difference between the two groups) using an HLM on the matched student groups. Success 
was defined as a grade of C or higher as determined by classroom instructors. 3 We ran three-level HLM 
analyses and estimated effects separately for Years 1 and 2. We used faculty as the second level and 
assigned the faculty ID of the Statway students to their matched comparisons for purposes of the 
analysis. Student and college were used as levels 1 and 3, respectively. Finally, we compared college- 
level course performance between the two groups in the subsequent year (the subsequent year was 


A grade of C- or higher was used for six colleges that employ a +/- grading system to define college math success. 
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defined as three consecutive terms, including summer, immediately after spring 2012), as depicted in 
the right panel of Figure 1. We defined student performance as accumulated units earned with a grade 
of C or higher. 4 We used HLM 7 (Raudenbush, Bryk, Cheong, Congdon, & du Toit, 2011) for all of the 
HLM analyses. 


4 A grade of C- or higher was used for six colleges that employ a +/- grading system to define college-level units 
earned. 
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RESULTS 


Propensity Score Matching 

To obtain propensity scores, we formulated a two-level Bernoulli model and estimated its model 
parameters using maximum likelihood via adaptive Gaussian quadrature, cp/j is the probability of student / 
enrolling in Statway in college j. Accordingly, q is the log-odds of this incident and formally expressed as: 

Level-1 Model (Student) 

Prob(5tV =1\6) = <fi 

ij J ij 

iog[0./(i - 0..)] = n.. 

u u ij 

n = 6+6 *(COVl ) + ... + 6 *(COV44 ) 

ij Oj lj ij' 44j if 

Level-2 Model (College) 


6 = 

Y 

43 

430, 

6 = 

Y 

44j 

440 


+ u 

44j 


where SI/I/ is a dummy variable indicating whether a given student was enrolled in Statway (coded as 1) 
or not (coded as 0), COI/is a covariate, and / and j denote student and college, respectively. We 
estimated one random slope, 6 44j , for a dummy variable indicating a math placement two levels below 
the college math level to account for some observed heterogeneity among colleges in this relationship. 
Using the matching procedure described above, we identified a total of 4549 comparison students 
matched to 928 Statway students for Year 1. Using the same matching procedure, we identified a total 
of 3583 comparison students matched to 771 Statway students for Year 2. 5 Tables 4 and 5 present the 
balance in propensity score for those students by college for Year 1 and Year 2, respectively. For both 
cohorts, there were no significant differences in mean propensity score between the Statway and 
matched students in any of the colleges. Also, Tables 2 and 3 show the descriptive statistics after 
matching for Year 1 and Year 2, respectively. 


5 As did in Year 1, we estimated one random slope for a math placement two levels below college. However, 
derived propensity scores were more varied than those from the fixed slope HLM model. Hence, we decided to use 
propensity scores from the fixed slope model for matching. 
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Statway Effectiveness 

To estimate differences in success rate, we formulated a three-level Bernoulli model 6 and estimated its 
model parameters using maximum likelihood via adaptive Gaussian quadrature, cp, Jk represents the 
probability that student / within faculty member j's class in college k successfully achieves college math 
credit. Correspondingly, r ) ijk is the corresponding log-odds of this outcome and formally expressed as: 


Level-1 Model (Student) 

Prob (CM A =1 1 7T ) = cp 

ijk jk ijk 

iog[tf>. . 7(1 - (P. )] = n... 

ijk ijk ijk 
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Level-3 Model (College) 
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where CMA and PS represent college math achievement (1 for successfully completed and 0 for not 
successfully completed) and a propensity score for additional adjustment. The key predictor of interest, 
SI/I/, is a dummy variable indicating whether the student was enrolled in Statway (coded as 1) or one of 
the matched comparisons (coded as 0). 

The results displayed in Table 6 indicate that on average, Statway students demonstrated significantly 
higher odds of success, 5.31, in college-level mathematics than the comparison students. This translated 
into the estimated probabilities of success of 54.43 percent for the Statway group and 18.36 percent for 
the comparison group. Additionally, we found variation among colleges in student success (Variance = 
0.239). Figure 2 shows that all but one college demonstrated greater success in Statway. Most colleges 
clustered around a line representing "triple the success rate", i.e., Statway students were three times 
more likely to succeed than their matched students. This is represented by the dotted line in Figure 2. 


6 We also ran a four-level model where level 1 is student, level 2 is matched student cluster, level 3 is faculty, and 
level 4 is college and obtained similar results for both Year 1 and Year 2. 
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The HLM results for Year 2 are displayed in Table 7. Consistent with Year 1 results, Statway students 
demonstrated significantly higher odds of success in college-level mathematics, 7.40, as compared to 
the matched students (55.26 percent vs. 14.30 percent as estimated probabilities of success). The 
Statway effect on student success also varied across colleges (Variance = 0.342). Again, as portrayed in 
Figure 3, all but one college clustered around the "triple the success rate" line. 

To estimate differences in college credits earned with a grade of C or higher in the subsequent year, we 
formulated a three-level Poisson model 7 and estimated its model parameters using penalized quasi- 
likelihood estimation. 8 A,y* represents the event rate that student / within faculty member j's class in 
college k successfully earns college credits in the following year. Thus, n ijk is the corresponding log of this 
event and formally expressed as: 

Level-1 Model (Student) 


E(CCE// k | TTj K ) — Xjjk 


IOg[A,y/J = n ijk 


n ijk = n 0jk + n 1Jk *(PS iJk ) + n 2 j k *(SW ijk ) 


Level-2 Model (Faculty) 


Tioj k = 6 oo k + r oj k 
Klj k = &io k 
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Level-3 Model (College) 

Book = Vooo + Uook 
610k - V100 
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where CCE represents accumulated college-level units earned with a grade of C or higher in the 
subsequent year. 9 The results displayed in Table 8 indicate that on average, Statway students were 
significantly more likely to earn college credits than the comparison students (with the higher event rate 
ratio, 1.37). The estimated accumulated credits were 5.57 vs. 4.08, respectively. We again found 
variation among colleges (Variance = 0.320). Figure 4 depicts that in the majority of colleges, Statway 
students earned more college credits in the subsequent year than the matched students. 


7 We also ran a four-level model and obtained similar results. 

8 FILM 7 applies penalized quasi-likelihood estimation to a three or higher level Poisson model. 

9 There were three quarter colleges, and accordingly, their college-level units were converted into semester units 
by dividing the units by 1.5. 
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Subgroup Analyses 

To examine possible differential effects of Statway by (a) gender and race/ethnicity subgroups and by 
(b) math placement levels, we formulated a three-level HLM similar to those described above. 10 In these 
subgroup analyses, however, we applied effect coding to the grouping variables in order to accurately 
capture the main and interaction effects on the outcome. As reference categories, female, White, and a 
math placement three or more levels below college were coded as -1. Data regarding the unknown 
gender status and the college math placement level were excluded from these HLM analyses. 

For the gender and race/ethnicity subgroup analyses, we formulated four models: (a) a base model 
(including a propensity score as covariate), (b) a main effect model (with the Statway, gender, and 
race/ethnicity main effects), (c) a two-way interaction effect model of Statway by gender and 
race/ethnicity, and (d) a three-way interaction effect model (i.e., Statway x gender x race/ethnicity). For 
the analyses with math placement levels, we formulated three models: (a) a base model, (b) a main 
effect model, and (c) a two-way interaction effect model of Statway by math placement level. 

For each set of the analyses, those models were hierarchically related to one another and subjected to 
likelihood ratio tests. Table 9 presents these results. * 11 We found significant main effects of gender and 
race/ethnicity for both Year 1 and Year 2 (x 2 (5) = 58.59, p < .001; x 2 (5) = 44.87, p < .001). We also 
obtained significant Statway by math placement level interaction effects for both Year 1 and Year 2 
(X 2 (3) = 28.13, p < .001; x 2 (5) = 17.27, p < .001). 

Consistent with the likelihood ratio test results, Tables 10 and 11 indicate significant main effects of 
Statway and race/ethnicity on the success rate. Figure 5 presents the estimated probabilities of success 
by gender and race/ethnicity. In general, Statway effects appear consistent for all subgroups of 
students. Tables 12 and 13 and Figure 6 break down the same results by math placement level, 
demonstrating the similar effects of Statway. The only exception was a somewhat smaller effect among 
Year 1 students who were placed one level below college level. It is worth mentioning that even after 
taking into account other main effects and interaction effects, the Statway main effect size was still the 
largest, as evidenced in the odds ratios of 2.27, 2.64, 2.25, and 2.69 (see Tables 10, 11, 12, and 13). 

Similarly, we examined whether Statway effects on the college credit accumulation are different across 
the aforementioned subgroups. These results are presented in Tables 14 and 15 for gender and 
race/ethnicity and math placement level, respectively. We found significant interaction effects of 
Statway for gender and race/ethnicity as well as for math placement subgroups. Figure 7 presents the 
estimated college credit accumulation by gender and race/ethnicity and math placement level. The 
upper panel illustrates positive effects of Statway for each major race/ethnicity group: Black, Hispanic, 
and White. The lower panel indicates that regardless of math placement levels, overall, Statway 
students performed better than the comparison students. The effect appears the largest among 
students who were placed two levels below college level. 


10 We also ran four-level HLM analyses and found similar results. 

11 Because the penalized quasi-likelihood approach was used for estimating effects on college credit accumulation, 
the likelihood ratio test was not conducted. 
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Sensitivity Analyses 

In general, Statway effects were strong and prevalent for all subgroups. The validity of these effects was 
based on an assumption of strongly ignorable treatment assignment. In other words, all relevant 
covariates were included in the propensity score analysis so that the bias due to unmeasured covariates 
could be ignored. Thus, we examined the sensitivity of the estimated Statway effects to possible 
confounding by unmeasured variables (Hong & Raudenbush, 2005, 2006; Lin, Psaty, & Kronmal, 1998; 
Rosenbaum, 1986). Given some unmeasured covariates (U), the Statway effect (<5) can be re-estimated 
by adjusting for some hypothesized hidden bias (y(E[L/ 3 ]-E[l/ 0 ])) as: 

6*=<5-|/(E[^]-E[U 0 ]) 

where y is the unmeasured covariates' association with the outcome and E[L/j]-E[L/ 0 ] is their association 
with treatment assignment (i.e., Statway or non-Statway enrollment). 

Adapting the approach of Hong and Raudenbush (2005, 2006), we operationally defined a proxy for y as 
a coefficient derived from a three-level model designed to predict the outcome with the same set of 
covariates used in the propensity score analysis and E[l/J-E[l/ 0 ] as the observed mean difference 
between the Statway and non-Statway groups on the corresponding covariate (for the set of covariates, 
see Tables 2 and 3). We then selected the largest positive value of the product of these two values as 
the largest possible bias 12 and obtained an adjusted Statway estimate (<5 ). Accordingly, we re-estimated 
Statway effects on the college math achievement in Year 1 and Year 2 and on the college credit 
accumulation in Year 1 and constructed 95 percent confidence intervals for each new estimate. 

Table 16 presents the original Statway effect estimate, its adjusted estimate, and the 95 percent 
confidence interval of the adjusted estimate for each student outcome. As can be seen in this table, with 
adjustments for the largest hidden bias, none of the confidence intervals for the new Statway effect 
estimates contained 0 or any negative values, thereby supporting the strong ignorability assumption. 
Thus, it is very unlikely that our general conclusion regarding the positive effects of Statway on the 
student outcomes has been influenced by the omission of unmeasured confounding factors. 


12 We used the sum of the product values for those requiring a set of dummy variables (e.g., race/ethnicity, math 
placement level). 
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DISCUSSION 

The current study sought to undertake a rigorous causal analysis of Statway's efficacy for community 
college students. To assess this effectiveness, we used a propensity score matching technique with a 
hierarchical linear modeling approach and formulated a more rigorous comparison group. Given the 
relatively small size of each Statway cohort compared to other developmental math students at each 
college, we were able to show a high degree of propensity score matching in 44 different indicators. 
Given the large size of the estimated effects, we conclude that there is robust evidence of Statway 
accelerating student success in acquiring college-level math credit. Not only did were we able to 
replicate our results across two Statway cohorts, but we were also able to replicate them across 
students within different gender and race/ethnicity groups as well as those within different math 
placement levels. Furthermore, our findings suggest that the Statway effect persists even after students 
complete the program: Statway students tended to accumulate more college credits with a grade of C or 
higher than their non-Statway counterparts. 

In conclusion, these overall Statway results are very promising. However, we did detect some variation 
in outcomes among colleges. Accordingly, we need to further investigate how local context conditions 
affect Statway performance and use this information in order to formulate improvement strategies. As 
follow-up data become available to us, we also want to examine post-Statway performance for the Year 
2 cohort to see if post-Statway success replicates across cohorts. Another direction for future research is 
to follow Statway students who transferred to a four-year university and examine their long-term 
performance. This analysis will further illuminate the dimensions and possible limitations of Statway's 
effectiveness. 
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Table 1 

Demographic Information of the First and Second Cohorts of Statway 



Year 1 

Year 2 

Mathematics Placement Level 

College Level 

4.30% 

4.60% 

1 level below college level 

17.60% 

14.40% 

2 levels below college level 

51.80% 

46.10% 

3 or more levels below college level 

26.30% 

34.90% 

Reading Placement Level 

College Level 

51.90% 

51.50% 

1 level below college level 

39.20% 

33.20% 

2 levels below college level 

7.10% 

10.30% 

3 levels below college level 

1.80% 

5.00% 

Gender 

Female 

59.90% 

59.80% 

Male 

40.10% 

40.20% 

Race/Ethnicity 

African American 

24.70% 

24.00% 

Hispanic 

33.10% 

32.50% 

Caucasian 

29.20% 

33.30% 

Other 

13.00% 

10.20% 

Home Language Growing Up 

English only 

55.50% 

60.30% 

English and another language 

32.40% 

28.50% 

Non-English language only 

12.10% 

11.20% 

Maternal Education 

Less than high school 

16.00% 

14.20% 

High school graduate or GED 

31.40% 

27.10% 

Some college but no degree 

24.30% 

27.40% 

2-year college degree 

8.00% 

12.00% 

4-year college degree 

12.80% 

14.00% 

Graduate or professional degree 

7.50% 

5.30% 


Note. Adapted from Strother et al. (2013) and Van Campen et al. (2013) 
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Table 2 


Descriptive Statistics of Covariates in the Two-Level Propensity Model - Year 1 


Covariate 

Non-Statway 
Before After 

matching matching 

% % 

Statway 

% 

Cohort 

First year* 

57 

43 

40 

Second year or older 

43 

57 

60 

Gender 

Female* 

57 

57 

58 

Male 

43 

43 

42 

Unknown 

0 

0 

0 

Race/Ethnicity 

Black 

21 

24 

25 

Hispanic 

37 

28 

29 

White* 

29 

32 

29 

Other 

8 

10 

11 

Unknown 

5 

6 

6 

Type of first-time student 

First-time college* 

82 

75 

74 

First-time transfer 

18 

25 

26 

Dual enrollment in a previous term 

Yes 

4 

4 

4 

No* 

87 

78 

78 

Unknown 

9 

18 

18 

Math placement level 

College Level 

6 

1 

2 

1 level below college level 

18 

17 

15 

2 levels below college level 

35 

43 

43 

3+ levels below college level* 

26 

21 

21 

Unknown 

15 

18 

19 

English placement level 

College level* 

29 

33 

32 

Developmental level 

47 

41 

41 

Unknown 

24 

26 

27 

Reading placement level 

College level* 

31 

33 

32 

Developmental level 

39 

33 

33 

Unknown 

30 

34 

35 
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Part time vs. Full time 


Fulltime* 54 56 54 

Part time 46 44 46 



M 

SD 

M 

SD 

M 

SD 

Age (in years) 

23.15 

7.32 

25.58 

9.78 

26.08 

10.25 

Prior course enrollment and performance 







College math units attempted 

0.05 

0.56 

0.06 

0.57 

0.06 

0.61 

College math units completed 

0.02 

0.34 

0.02 

0.28 

0.02 

0.31 

College math courses attempted 

0.02 

0.17 

0.02 

0.18 

0.02 

0.20 

College math courses completed 

0.01 

0.10 

0.00 

0.08 

0.01 

0.10 

Developmental math units attempted 

2.45 

4.11 

3.52 

5.40 

3.62 

5.51 

Developmental math units completed 

1.39 

2.76 

1.93 

3.45 

1.95 

3.74 

Developmental math courses attempted 

0.72 

1.15 

1.00 

1.45 

1.02 

1.45 

Developmental math courses completed 

0.41 

0.76 

0.56 

0.94 

0.57 

0.99 

College non-math units attempted 

7.57 

13.25 

12.92 

19.23 

13.90 

20.18 

College non-math units completed 

5.58 

10.78 

9.74 

15.83 

10.71 

16.63 

College non-math courses attempted 

2.53 

4.29 

4.21 

6.00 

4.52 

6.11 

College non-math courses completed 

1.88 

3.49 

3.19 

4.88 

3.51 

5.14 

Developmental non-math units attempted 

2.00 

5.11 

2.43 

6.30 

2.45 

6.79 

Developmental non-math units completed 

1.59 

4.33 

2.04 

5.51 

2.02 

5.76 

Developmental non-math courses attempted 

0.62 

1.49 

0.73 

1.78 

0.76 

1.98 

Developmental non-math courses completed 

0.49 

1.25 

0.62 

1.55 

0.63 

1.73 

College STEM courses attempted 

0.31 

1.03 

0.45 

1.23 

0.44 

1.15 

College STEM courses completed 

0.21 

0.82 

0.28 

0.89 

0.28 

0.86 

College non-STEM courses attempted 

2.21 

3.84 

3.76 

5.52 

4.09 

5.69 

College non-STEM courses completed 

1.65 

3.13 

2.90 

4.54 

3.22 

4.85 

GPA of college STEM courses 

2.01 

1.35 

1.97 

1.35 

2.05 

1.32 

GPA of college non-STEM courses 

2.36 

1.13 

2.46 

1.05 

2.51 

1.03 

Missing on college STEM GPA 

0.84 

0.36 

0.78 

0.41 

0.77 

0.42 

Missing on college non-STEM GPA 

0.55 

0.50 

0.46 

0.50 

0.42 

0.50 


Note. Terms with were used as reference categories (coded as 0, otherwise 1) when formulating 
dummy variables. First year under Cohort was defined as summer/fall enrollment in a given college for 
the first time in 2011 or 2012 for non-Statway or Statway. Part time vs. Full time status was based on 
fall 2011 or 2011 enrollment for non-Statway or Statway, with 12 or more units considered as full time. 
Age was computed by subtracting a birth year from 2011 or 2012 for non-Statway or Statway; in the 
current analyses, we centered Age around age 18. "Completed" was defined as course credit attained 
with a grade of C or higher (C- or higher if a college employs a +/- grading system) or Pass for 
developmental courses. 
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Table 3 


Descriptive Statistics of Covariates in the Two-Level Propensity Model - Year 2 



Non-Statway 

Statway 

Covariate 

Before 

matching 

After 

matching 



% 

% 

% 

Cohort 

First year* 

51 

33 

33 

Second year or older 

49 

67 

67 

Gender 

Female* 

58 

61 

60 

Male 

42 

39 

40 

Unknown 

0 

0 

0 

Race/Ethnicity 

Black 

23 

25 

25 

Hispanic 

38 

30 

29 

White* 

25 

31 

32 

Other 

8 

9 

9 

Unknown 

6 

5 

5 

Dual enrollment in a previous term 

Yes 

4 

2 

2 

No* 

85 

79 

76 

Unknown 

11 

19 

22 

Math placement level 

College Level 

4 

2 

2 

1 level below college level 

15 

8 

6 

2 levels below college level 

31 

31 

29 

3+ levels below college level* 

31 

21 

24 

Unknown 

19 

38 

39 

English placement level 

College level* 

27 

19 

19 

Developmental level 

45 

30 

31 

Unknown 

28 

51 

50 

Reading placement level 

College level* 

29 

24 

25 

Developmental level 

37 

25 

26 

Unknown 

34 

51 

49 

Part time vs. Full time 

Full time* 

48 

52 

54 

Part time 

52 

48 

46 
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M 

SD 

M 

SD 

M 

SD 

Age (in years) 

23.54 

7.34 

27.00 

9.56 

27.34 

9.79 

Prior course enrollment and performance 







College math units attempted 

0.05 

0.50 

0.10 

0.73 

0.13 

0.85 

College math units completed 

0.02 

0.25 

0.02 

0.30 

0.02 

0.33 



% 


% 


% 

College math courses attempted 







0* 


99 


97 

96 

1 


1 


2 


3 

2 or more 


0 


1 


1 


M 

SD 

M 

SD 

M 

SD 

College math courses completed 

0.01 

0.07 

0.01 

0.08 

0.01 

0.08 

Developmental math units attempted 

2.71 

3.97 

3.23 

4.35 

3.20 

4.50 

Developmental math units completed 

1.54 

2.60 

1.47 

2.60 

1.46 

2.40 

Developmental math courses attempted 

0.83 

1.20 

0.98 

1.31 

0.95 

1.32 

Developmental math courses completed 

0.47 

0.77 

0.45 

0.78 

0.44 

0.71 

College non-math units attempted 

8.63 

13.41 

13.50 

16.48 

14.35 

17.47 

College non-math units completed 

6.30 

10.72 

10.45 

13.66 

11.35 

14.56 

College non-math courses attempted 

2.90 

4.42 

4.47 

5.40 

4.69 

5.67 

College non-math courses completed 

2.11 

3.46 

3.44 

4.37 

3.66 

4.56 

Developmental non-math units attempted 

2.12 

4.10 

1.73 

3.82 

1.68 

3.79 

Developmental non-math units completed 

1.75 

3.66 

1.47 

3.49 

1.47 

3.50 

Developmental non-math courses attempted 

0.68 

1.31 

0.53 

1.17 

0.52 

1.16 

Developmental non-math courses completed 

0.56 

1.16 

0.45 

1.06 

0.45 

1.05 

College STEM courses attempted 

0.36 

0.84 

0.43 

0.91 

0.43 

0.89 

College STEM courses completed 

0.24 

0.62 

0.28 

0.67 

0.27 

0.65 

College non-STEM courses attempted 

2.45 

3.83 

3.93 

4.77 

4.16 

5.02 

College non-STEM courses completed 

1.82 

3.12 

3.10 

4.02 

3.34 

4.23 

GPA of college STEM courses 

2.11 

1.35 

2.08 

1.36 

2.04 

1.40 

GPA of college non-STEM courses 

2.30 

1.13 

2.52 

1.03 

2.59 

0.98 

Missing on college STEM GPA 

0.79 

0.41 

0.75 

0.43 

0.76 

0.43 

Missing on college non-STEM GPA 

0.53 

0.50 

0.42 

0.49 

0.42 

0.49 


Note. Terms with were used as reference categories (coded as 0, otherwise 1) when formulating 
dummy variables. First year under Cohort was defined as summer/fall enrollment in a given college for the 
first time in 2011 or 2012 for non-Statway or Statway. Part time vs. Full time status was based on fall 2011 
or 2011enrollment for non-Statway or Statway, with 12 or more units considered as full time. Age was 
computed by subtracting a birth year from 2011 or 2012 for non-Statway or Statway; in the current 
analyses, we centered Age around age 18. "Completed" was defined as course credit attained with a grade 
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of C or higher (C- or higher if a college employs a +/- grading system) or Pass for developmental courses. 
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Table 4 

Balance in Logit of the Propensity Score for non-Statway and Statway Students - Year 1 


College 



Non-Statway 




Statway 


Sample before matching 

Sample after matching 




N 

M 

SD 

N 

M 

SD 

N 

M 

SD 

1 

3463 

-3.98 

0.81 

477 

-3.18 

0.92 

97 

-3.13 

0.96 

2 

637 

-3.25 

0.72 

171 

-2.83 

0.75 

36 

-2.72 

0.90 

3 

3857 

-4.15 

0.61 

385 

-3.65 

0.62 

77 

-3.65 

0.62 

4 

2270 

-3.74 

0.51 

320 

-3.53 

0.54 

65 

-3.50 

0.61 

5 

2610 

-3.93 

0.48 

286 

-3.46 

0.67 

60 

-3.40 

0.74 

6 

1214 

-3.16 

0.79 

341 

-2.80 

0.58 

70 

-2.76 

0.64 

7 

2408 

-4.12 

0.82 

254 

-3.83 

0.62 

51 

-3.82 

0.64 

8 

1451 

-3.71 

0.79 

228 

-3.09 

0.64 

48 

-3.00 

0.72 

9 

2243 

-3.70 

0.65 

341 

-3.34 

0.64 

70 

-3.28 

0.73 

10 

3975 

-4.61 

0.54 

240 

-4.16 

0.50 

48 

-4.17 

0.49 

11 

8623 

-5.67 

1.07 

255 

-4.85 

0.69 

51 

-4.85 

0.69 

12 

6779 

-4.76 

0.58 

340 

-4.22 

0.57 

69 

-4.16 

0.62 

13 

4763 

-5.40 

0.50 

110 

-5.10 

0.49 

22 

-5.10 

0.50 

14 

8955 

-5.21 

0.54 

280 

-4.69 

0.71 

56 

-4.69 

0.72 

15 

2970 

-4.76 

0.78 

171 

-4.00 

0.72 

35 

-3.96 

0.76 

16 

714 

-3.26 

0.59 

171 

-2.91 

0.55 

36 

-2.83 

0.68 

17 

1102 

-3.84 

0.88 

179 

-3.18 

1.01 

37 

-3.12 

1.13 

Total 

58034 

-4.66 

1.02 

4549 

-3.65 

0.93 

928 

-3.60 

0.98 
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Table 5 

Balance in Logit of the Propensity Score for non-Statway and Statway Students - Year 2 


Non-Statway Statway 


Sample before matching Sample after matching 


College 

N 

M 

SD 

N 

M 

SD 

N 

M 

SD 

1 

3517 

-3.83 

0.97 

626 

-2.94 

0.93 

126 

-2.92 

0.94 

2 

580 

-3.78 

1.02 

115 

-3.30 

1.02 

23 

-3.31 

1.02 

3 

3969 

-4.33 

0.96 

418 

-3.17 

1.10 

91 

-3.01 

1.20 

5 

2905 

-4.34 

0.86 

280 

-3.59 

0.68 

56 

-3.59 

0.68 

6 

987 

-3.36 

0.79 

241 

-2.64 

0.64 

50 

-2.63 

0.62 

7 

2129 

-4.46 

0.72 

170 

-3.50 

0.31 

34 

-3.50 

0.32 

8 

1618 

-3.81 

0.71 

238 

-2.98 

0.82 

51 

-2.86 

0.94 

9 

1976 

-3.82 

0.73 

310 

-2.80 

0.60 

62 

-2.80 

0.59 

10 

3902 

-5.07 

0.62 

145 

-4.58 

0.69 

30 

-4.51 

0.80 

12 

6999 

-5.02 

0.90 

365 

-4.38 

1.09 

73 

-4.38 

1.10 

13 

4613 

-5.73 

0.69 

90 

-4.93 

0.69 

18 

-4.93 

0.71 

14 

9994 

-5.95 

0.78 

175 

-5.48 

1.05 

35 

-5.48 

1.06 

15 

3317 

-5.25 

0.87 

130 

-4.33 

0.78 

26 

-4.33 

0.79 

16 

789 

-2.99 

0.79 

104 

-2.09 

0.80 

60 

-1.89 

0.91 

17 

1088 

-4.03 

1.13 

176 

-3.62 

1.20 

36 

-3.56 

1.28 

Total 

48383 

-4.90 

1.17 

3583 

-3.47 

1.19 

771 

-3.34 

1.26 
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Table 6 

Model-Based Estimation of Statway Effect on College Math Achievement - Year 1 


Fixed effect 

Coefficient 

SE 

t 

p-value 

Odds ratio 

Intercept 

-1.67 

0.13 

-13.06 

<0.001 

0.19 

Propensity score 

0.17 

0.05 

3.37 

<0.001 

1.19 

Statway effect 

1.67 

0.08 

20.84 

<0.001 

5.31 

Random effect 

Variance 

df 

/ 

p-value 


Level 3 (college) 

0.239 

16 

153.32 

< 0.001 


Level 2 (faculty) 

0.014 

24 

31.43 

0.142 



Note. The df's, x 2 statistics, and p-values of the random effects are for variance estimates derived 
from penalized quasi-likelihood estimation but reported here to indicate approximate significance 
levels. 
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Table 7 

Model-Based Estimation of Statway Effect on College Math Achievement - Year 2 


Fixed effect 

Coefficient 

SE 

t 

p-value 

Odds ratio 

Intercept 

-1.81 

0.16 

-11.27 

<0.001 

0.16 

Propensity score 

0.02 

0.05 

0.35 

0.726 

1.02 

Statway effect 

2.00 

0.09 

22.02 

<0.001 

7.40 

Random effect 

Variance 

df 

/ 

p-value 


Level 3 (college) 

0.342 

14 

179.22 

<0.001 


Level 2 (faculty) 

0.000 

21 

17.53 

>0.500 



Note. The df's,^ statistics, and p-values of the random effects are for variance estimates derived 
from penalized quasi-likelihood estimation but reported to indicate approximate significance levels. 
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Table 8 

Model-Based Estimation of Statway Effect on Accumulated College Credits Earned in the Subsequent 
Year - Year 1 


Fixed effect 

Coefficient 

SE 

t 

p-value 

Event rate ratio 

Intercept 

1.51 

0.14 

10.51 

<0.001 

4.52 

Propensity score 

-0.10 

0.03 

-3.09 

0.002 

0.90 

Statway effect 

0.31 

0.11 

2.89 

0.004 

1.37 

Random effect 

Variance 

df 

X2 

p-value 


Level 3 (college) 

0.320 

16 

373.82 

<0.001 


Level 2 (faculty) 

0.032 

24 

367.71 

<0.001 
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Table 9 


Model Comparison of Subgroup Analyses on College Math Achievement 




Year 1 



Year 2 


Gender and Race/Ethnicity 

/ 

df 

p-value 

/ 

df 

p-value 

Base vs. Main 

58.59 

5 

<o.oo: 

44.87 

5 

<0.001 

Main vs. Two-way 

12.59 

9 

o.i8; 

15.18 

9 

0.086 

Two-way vs. Three way 

3.16 

4 

0.53: 

3.50 

4 

0.477 

Math placement level 

/ 

df 

p-value 

/ 

df 

p-value 

Base vs. Main 

27.43 

3 

<o.oo: 

25.54 

3 

<0.001 

Main vs. Two-way 

28.13 

3 

<o.oo: 

17.27 

3 

0.001 


Note. % 2 and df reflect differences in deviance statistics and the number of estimated parameters 
between two models, respectively. 


© 2014 CARNEGIE FOUNDATION FOR THE ADVANCEMENT OF TEACHING 


Community College Pathways' Program Success 


Table 10 

Model-Based Estimation of Statway, Gender, and Race/Ethnicity Effects on College Math Achievement - 
Year 1 


Fixed effect 

Coefficient 

SE 

t 

p-value 

Odds ratio 

Intercept 

-1.40 

0.14 

-10.22 

<0.001 

0.25 

Propensity score 

0.21 

0.05 

4.00 

<0.001 

1.23 

Statway 

0.82 

0.05 

16.71 

<0.001 

2.27 

Gender 

-0.09 

0.05 

-1.87 

0.062 

0.91 

Black 

-0.50 

0.09 

-5.64 

<0.001 

0.60 

Hispanic 

-0.08 

0.08 

-1.00 

0.318 

0.92 

Other 

0.13 

0.11 

1.24 

0.216 

1.14 

Unknown 

0.26 

0.13 

1.91 

0.057 

1.29 

Statway x Gender 

-0.01 

0.05 

-0.19 

0.849 

0.99 

Statway x Black 

0.10 

0.08 

1.23 

0.217 

1.11 

Statway x Hispanic 

-0.01 

0.08 

-0.08 

0.939 

0.99 

Statway x Other 

-0.22 

0.11 

-2.09 

0.037 

0.80 

Statway x Unknown 

0.04 

0.13 

0.30 

0.762 

1.04 

Gender x Black 

-0.01 

0.08 

-0.13 

0.901 

0.99 

Gender x Hispanic 

-0.11 

0.08 

-1.47 

0.143 

0.89 

Gender x Other 

0.06 

0.11 

0.53 

0.595 

1.06 

Gender x Unknown 

0.05 

0.13 

0.37 

0.711 

1.05 

SW x Gender x Black 

0.01 

0.08 

0.10 

0.918 

1.01 

SW x Gender x Hispanic 

0.04 

0.08 

0.50 

0.618 

1.04 

SW x Gender x Other 

-0.04 

0.10 

-0.34 

0.736 

0.97 

SW x Gender x Unknown 

0.11 

0.13 

0.80 

0.424 

1.11 

Random effect 

Variance 

df 

X2 

p-value 


Level 3 (college) 
Level 2 (faculty) 

0.290 

0.007 

16 

24 

206.99 

29.07 

<0.001 

0.217 



Note. The df's,^ statistics, and p-values of the random effects are for variance estimates derived from 
penalized quasi-likelihood estimation but reported here to indicate approximate significance levels. 
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Table 11 

Model-Based Estimation of Statway, Gender, and Race/Ethnicity Effects on College Math Achievement - 
Year 2 


Fixed effect 

Coefficient 

SE 

t 

p-value Odds ratio 

Intercept 

-1.47 

0.15 

-9.66 

<0.001 

0.23 

Propensity score 

0.03 

0.05 

0.75 

0.451 

1.03 

Statway 

0.97 

0.06 

16.31 

<0.001 

2.64 

Gender 

-0.02 

0.06 

-0.38 

0.707 

0.98 

Black 

-0.49 

0.10 

-4.77 

<0.001 

0.61 

Hispanic 

-0.04 

0.09 

-0.45 

0.650 

0.96 

Other 

0.22 

0.13 

1.67 

0.095 

1.24 

Unknown 

0.03 

0.17 

0.20 

0.841 

1.03 

Statway x Gender 

-0.06 

0.06 

-1.05 

0.293 

0.94 

Statway x Black 

0.13 

0.10 

1.35 

0.178 

1.14 

Statway x Hispanic 

-0.06 

0.09 

-0.71 

0.477 

0.94 

Statway x Other 

-0.18 

0.13 

-1.40 

0.161 

0.84 

Statway x Unknown 

0.05 

0.17 

0.29 

0.773 

1.05 

Gender x Black 

-0.04 

0.10 

-0.40 

0.689 

0.96 

Gender x Hispanic 

0.08 

0.09 

0.85 

0.393 

1.08 

Gender x Other 

-0.05 

0.13 

-0.37 

0.714 

0.95 

Gender x Unknown 

0.01 

0.16 

0.05 

0.960 

1.01 

SW x Gender x Black 

-0.12 

0.10 

-1.26 

0.209 

0.88 

SW x Gender x Hispanic 

-0.10 

0.09 

-1.14 

0.254 

0.90 

SW x Gender x Other 

0.07 

0.13 

0.57 

0.569 

1.08 

SW x Gender x Unknown 

0.22 

0.16 

1.36 

0.174 

1.25 

Random effect 

Variance 

df 

X2 

p-value 


Level 3 (college) 
Level 2 (faculty) 

0.311 

0.000 

14 

21 

162.03 

18.11 

<0.001 

>0.500 



Note. The df's,^ statistics, and p-values of the random effects are for variance estimates derived from 
penalized quasi-likelihood estimation but reported here to indicate approximate significance levels. 
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Table 12 

Model-Based Estimation of Statway and Math Placement Level Effects on College Math Achievement - 
Year 1 


Fixed effect 

Coefficient 

SE 

t 

p-value 

Odds ratio 

Intercept 

-1.41 

0.13 

-10.61 

<0.001 

0.24 

Propensity score 

0.20 

0.05 

3.93 

<0.001 

1.23 

Statway 

0.81 

0.04 

18.68 

<0.001 

2.25 

1 level below college level 

0.16 

0.10 

1.69 

0.092 

1.18 

2 levels below college level 

0.03 

0.07 

0.43 

0.668 

1.03 

Unknown 

0.02 

0.08 

0.25 

0.806 

1.02 

Statway x 1 level below 

-0.40 

0.08 

-4.96 

<0.001 

0.67 

Statway x 2 levels below 

0.17 

0.06 

2.80 

0.005 

1.19 

Statway x Unknown 

0.04 

0.08 

0.54 

0.592 

1.04 

Random effect 

Variance 

df 

X2 

p-value 


Level 3 (college) 
Level 2 (faculty) 

0.275 

0.000 

16 

24 

201.37 

26.95 

<0.001 

0.306 



Note. The df's,^ statistics, and p-values of the random effects are for variance estimates derived 
from penalized quasi-likelihood estimation but reported here to indicate approximate significance 
levels. 
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Table 13 

Model-Based Estimation of Statway and Math Placement Level Effects on College Math Achievement - 
Year 2 


Fixed effect 

Coefficient 

SE 

t 

p-value 

Odds ratio 

Intercept 

-1.50 

0.17 

-8.77 

<0.001 

0.22 

Propensity score 

0.06 

0.05 

1.11 

0.269 

1.06 

Statway 

0.99 

0.06 

17.78 

<0.001 

2.69 

1 level below college level 

0.43 

0.14 

3.09 

0.002 

1.54 

2 levels below college level 

-0.18 

0.09 

-2.05 

0.041 

0.84 

Unknown 

-0.05 

0.09 

-0.57 

0.566 

0.95 

Statway x 1 level below 

-0.24 

0.13 

-1.88 

0.060 

0.79 

Statway x 2 levels below 

-0.17 

0.08 

-2.13 

0.034 

0.84 

Statway x Unknown 

0.11 

0.08 

1.42 

0.157 

1.12 

Random effect 

Variance 

df 

X2 

p-value 


Level 3 (college) 
Level 2 (faculty) 

0.401 

0.000 

14 

21 

185.78 

14.23 

<0.001 

>0.500 



Note. The df's,^ statistics, and p-values of the random effects are for variance estimates derived from 
penalized quasi-likelihood estimation but reported here to indicate approximate significance levels. 
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Table 14 

Model-Based Estimation of Statway, Gender, and Race/Ethnicity Effects on Accumulated College 
Credits Earned in the Subsequent Year - Year 1 


Fixed effect 

Coefficient 

SE 

t 

p-value 

Rate ratio 

Intercept 

1.54 

0.14 

10.97 

<0.001 

4.66 

Propensity score 

-0.08 

0.01 

-9.41 

<0.001 

0.92 

Statway 

0.13 

0.01 

13.81 

<0.001 

1.13 

Gender 

-0.01 

0.01 

-1.52 

0.127 

0.99 

Black 

-0.22 

0.02 

-13.30 

<0.001 

0.80 

Hispanic 

0.02 

0.01 

1.46 

0.144 

1.02 

Other 

0.10 

0.02 

5.05 

<0.001 

1.11 

Unknown 

0.04 

0.03 

1.36 

0.173 

1.04 

Statway x Gender 

0.04 

0.01 

4.88 

<0.001 

1.05 

Statway x Black 

0.16 

0.02 

10.13 

<0.001 

1.17 

Statway x Hispanic 

-0.01 

0.01 

-0.78 

0.434 

0.99 

Statway x Other 

-0.18 

0.02 

-9.21 

<0.001 

0.83 

Statway x Unknown 

-0.03 

0.03 

-1.35 

0.177 

0.97 

Gender x Black 

-0.02 

0.02 

-1.48 

0.140 

0.98 

Gender x Hispanic 

-0.04 

0.01 

-2.67 

0.008 

0.96 

Gender x Other 

-0.11 

0.02 

-5.58 

<0.001 

0.89 

Gender x Unknown 

0.16 

0.03 

6.16 

<0.001 

1.17 

SW x Gender x Black 

0.01 

0.02 

0.94 

0.350 

1.01 

SW x Gender x Hispanic 

-0.06 

0.01 

-4.32 

<0.001 

0.94 

SW x Gender x Other 

-0.11 

0.02 

-5.58 

<0.001 

0.89 

SW x Gender x Unknown 

0.15 

0.03 

5.99 

<0.001 

1.17 

Random effect 

Variance 

df 

X2 

p-value 


Level 3 (college) 
Level 2 (faculty) 

0.319 

0.033 

16.00 

24.00 

364.03 

394.34 

<0.001 

<0.001 
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Table 15 

Model-Based Estimation of Statway and Math Placement Level Effects on Accumulated College Credits 
Earned in the Subsequent Year - Year 1 


Fixed effect 

Coefficient 

SE 

t 

p-value 

Rate ratio 

Intercept 

1.55 

0.14 

11.00 

<0.001 

4.69 

Propensity score 

-0.08 

0.01 

-9.56 

<0.001 

0.92 

Statway 

0.15 

0.01 

19.31 

<0.001 

1.16 

1 level below college level 

0.27 

0.02 

16.31 

<0.001 

1.31 

2 levels below college level 

0.07 

0.01 

5.80 

<0.001 

1.08 

Unknown 

-0.20 

0.02 

-12.73 

<0.001 

0.82 

Statway x 1 level below 

-0.05 

0.01 

-3.78 

<0.001 

0.95 

Statway x 2 levels below 

0.08 

0.01 

7.52 

<0.001 

1.09 

Statway x Unknown 

-0.05 

0.02 

-3.33 

<0.001 

0.95 

Random effect 

Variance 

df 

X2 

p-value 


Level 3 (college) 
Level 2 (faculty) 

0.322 

0.029 

16 

24 

403.32 

328.72 

<0.001 

<0.001 
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Table 16 

Sensitivity Analyses on Statway Effect 



Original 

estimate 

Adjusted 

estimate 

95% Cl 

Year 1 - College math achievement 

1.67 

1.50 

[1.34, 1.65] 

Year 2 - College math achievement 

2.00 

1.97 

[1.79, 2.15] 

Year 1 - College credit accumulation 

0.31 

0.22 

[0.01, 0.43] 
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Start Term 
(Dev Math) 


Fall 10 



End Term 
(College Math) 


Following Year 
(College Level Units) 


Spring 12 





Sul2 

Fal2 

Spl3 





Success rate 
C or better 


Accumulated college- 
level units earned with C 
or better 



( \ 


r \ 

Success rate 


Accumulated college- 

C or better 

* 

level units edrneci wiin v, 
or better 

V ) 


^ J 


Figure 1. Study design. For Statway students, data were based on registrar reports from Statway 
classrooms. For non-Statway (Non-SW) students, results were based on whichever college-level math 
course, if any, was completed by spring 2012. If more than one course was completed during the two 
years, data from the course (minimum 3 units) with a higher grade were included. A grade of C- or 
higher was employed for six colleges that use a +/- grading system to define college math success and 
college-level units earned. Sul2, Fal2, and Spl3 represent summer 2012, fall 2012, and spring 2013, 
respectively. 
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Non-Statway 


Figure 2. Comparative success rates by college - Year 1. The success rates of the matched comparison 
students are represented along the x-axis, and those of the Statway student are represented along the 
y-axis. For ease of interpretation, two reference lines are provided. The 45 degree solid line indicates no 
difference in outcome (Statway vs. Non-Statway). The dotted line represents "triple the success" rate of 
Statway students against their Non-Statway counterparts. The numeric values represent pseudo-college 
IDs. 
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Non-Statway 


Figure 3. Comparative success rates by college - Year 2. The success rates of the matched comparison 
students are represented along the x-axis, and those of the Statway student are represented along the 
y-axis. For ease of interpretation, two reference lines are provided. The 45 degree solid line indicates no 
difference in outcome (Statway vs. Non-Statway). The dotted line represents "triple the success" rate of 
Statway students against their Non-Statway counterparts. The numeric values represent pseudo-college 
IDs. 
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Non-Statway 


Figure 4. Comparative college-level credits accumulated in the subsequent year by college - Year 1. The 
college units of the matched comparison students are represented along the x-axis, and those of the 
Statway student are represented along the y-axis. For ease of interpretation, the 45 degree solid line 
indicates no difference in outcome (Statway vs. Non-Statway). The numeric values represent pseudo- 
college IDs. 
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Year 1 



Non-Statway ■ Statway 


Year 2 



Non-Statway ■ Statway 


Figure 5. Model-based success rates by gender and race/ethnicity. 
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1 level below 2 levels below 3+ levels below Unknown 

Non-Statway ■ Statway 

Year 2 



1 level below 2 levels below 3+ levels below Unknown 

Non-Statway ■ Statway 

Figure 6. Model-based success rates by math placement level. 
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By gender and race/ethnicity - Year 1 
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Figure 7. Model-based college-level credits accumulated in the subsequent year by gender and 
race/ethnicity (upper panel) and math placement level (lower panel) - Year 1. 
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